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A USER SUPPORT APPARATUS AND SYSTEM USING AGENTS 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates to a user support 
technique, and it particularly relates to a user support 
system that supports users' processes such as an operation 
and an information retrieval using agents. 

2. Description of the Related Art 

With the widespread use of the Internet and mobile 
phones, more and more people are using personal computers 
and various types of information terminals. In recent 
years, sales of electronic equipment such as personal 
computers have been greatly increasing, and as a result 
beginners or persons who lack computer literacy have 
rapidly increased in number. With the remarkable 
improvement in CPU power, memory capacity and graphical 
user interface or GUI, such novices have been provided with 
a method which allows them to operate their equipment 
easily. Without such aids as GUI and the like, the novice 
is unable to utilize many advanced functions of the 
equipment and to access necessary information properly. 

As one such user support technology, an agent 
technology is known and is in use. An agent is basically a 
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personified character that appears on a computer display, 
obtains a user's utterance, and transfers the utterance to 
the computer. The result of processes in the computer is 
presented to the user as an utterance from the agent. With 
the presence of the agent, the inexperienced user can be 
relieved of a sense of awkwardness when he/she works with 
the computer and can feel as though the computer is talking 
to him/her. 

On the other hand, it demands a very great effort for 
a system administrator to manage and refine an agent system 
properly. As the user is a human being, the range of 
his/her utterances is almost infinite. Although it is 
impossible to anticipate all of these utterances and 
prepare for real utterances produced by the user, it is 
possible that the user may expect that the agent can 
accurately understand any utterances he/she might make and 
give quick and proper responses to them. Even if a 
substantial amount of the user utterances is anticipated, 
it is a very hard task to search and identify the words 
expressed by the user in a very short response time close 
to real-time. Moreover, as the number of the anticipated 
utterances increases, the load on the server that controls 
the agent also increases, as it must handle a larger amount 
of data. 

SUMMARY OF THE INVENTION 
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The present invention has been made with a view to 
the above-mentioned problems, and an object thereof is to 
provide a user support technology by means of which an 
5 agent can give a quick response to a wide range of user 
utterances and requests. Another object of the present 
invention is to provide an agent technology by which the 
responses of agents can be continuously and effectively 
enhanced in respect to accuracy and flexibility. 

10 According to one aspect of the present invention, a 

user support system is provided. The system comprises a 
first block which has an electronic collection of user 
utterances, and identifies a content of a given user 
utterance, and a second block which has an electronic 

15 collection of action patterns for an agent for responding 

to user utterances, and enables the agent to respond to the 
user utterances. The user utterance collection and the 
agent action collection are separately constructed by 
configuring the first block and the second block separately. 

20 In this configuration the user utterance collection and the 
agent action collection can be accessed independently, and 
thereby the process of identifying the user utterances and 
the process of determining the agent responses can be 
executed in parallel. 

25 The agent is not always a visible character but the 

agent here may also means a user support program that is 
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itself invisible from the user or other functions such as a 
back-end process in the system. The action patterns of the 
agent include the agent utterances, images, behaviors, or 
any other processes related to supporting users. The 
utterance of the user and the agent is not only made in a 
voice, but also given in text data. The utterance may 
include oral or spoken words or sentences that can be 
converted to text data by speech recognition. 

The system may include a plurality of the second 
blocks, and each of the second blocks may offer a 
specialized service to the user. For instance, the first 
block and the plurality of the second block may be 
configured as different nodes in a network and communicate 
with the user via the network. In this case, user may be a 
client and the first and second blocks may be servers, and 
thus a client -server system can be configured. 

The first block may be an entrance or portal server 
to identify the user utterances, and an appropriate one of 
the second blocks may be selected according to the content 
of the identified user utterance. The second blocks may be 
provided for each service category, such as news, fortune 
telling, travel, cooking, business, health and so on. In 
this case, since each second block has a specific theme, 
the agent on each second block can be easily maintained and 
refined. In addition, since the utterances on different 
topics are processed on different nodes, the system load 
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can be distributed and balanced among the nodes , 

The first block may include an utterance search unit 
which searches the utterance of the user in the user 
utterance collection, and a reporting unit which notifies a 
system administrator when the user utterance is not found 
in the user utterance collection. The administrator can 
answer to the user or revise the user utterance collection 
and the agent action collection. 

The system may further comprise a recording unit that 
obtains a record of the user's access to the system. The 
second block may chooses one from a plurality of choices of 
the actions of the agent to respond to the user utterance 
depending on a situation of the user's access. 

The first block may further include an index storing 
unit that stores an index of contents of the user utterance 
collection. The search unit can initially perform an 
index- search for the given user utterance to narrow the 
search scope and the search speed can be improved. 

The search unit may perform a full text search for 
the user utterance. The full text search here means that 
the user utterance is matched with all utterances 
registered in the user utterance collection and compared on 
a full text basis. Although the full text search can be 
performed independently from the index search, the index 
search may be preferably performed to narrow the search 
scope before the full text search. In other words, the 
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system designer can register many user utterances without 
caring about their similarity and as a result the given 
user utterance could be easily searched by the full text 
search. By using the full text search to identify the user 
utterances, user's intention can be identified with high 
accuracy and the agent's response can be very accurate. 
Thus, the index search and the full text search can allow 
the volume of the user utterance collection to increase 
without sacrificing the system performance. 

Still another object of the present invention is to 
provide a translation system as a communication tool with 
great convenience and excellent performance. 

According to one aspect of the present invention, a 
translation system is provided. The system comprises a 
first block which has an electronic collection of user 
utterances, and identifies a content of a given user 
utterance, and a second block which has an electronic 
dictionary file for translating the user utterance, and 
provides the user with an expression corresponding to the 
utterance in another language. The first block and the 
second block are configured as different nodes accessing a 
network so that the user utterance collection and the 
dictionary file are separately constructed. The first 
block may be called an entrance server since it receives 
the user utterance, while the second block may be called a 
translation server since it is in charge of the translation. 
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The translation server may be divided into a plurality of 
servers and each server may deal with each specialized 
field. For instance, the servers may be separately 
configured for each of topics such as scientific technology, 
daily conversation, politics and economy, or the like. The 
entrance server may judge which server is most appropriate 
to the user utterance. 

In order to distribute the processes of identifying 
and translating the user utterance, the first and the 
second blocks are separately configured as different 
network nodes. The first and the second blocks may be 
configured as servers that the user terminal accesses. 
Such distributed processing can improve the overall 
performance of the system. Such distributed system is easy 
to maintain and enhance. If the translation servers are 
provided for each specialized field, each of the servers 
can be maintained independently. 

The corresponding expression is, for instance, an 
English translation "Good morning" of a Japanese utterance 
"Ohayo" . The system may include a target language setting 
unit that enables the user to sets a translation language 
or a target language . 

The first block may include an utterance search unit 
which searches the utterance of the user in the user 
utterance collection, and a reporting unit which notifies a 
system administrator when the user utterance is not found 
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in the user utterance collection. Thereby the 
administrator can revise the user utterance collection and 
the dictionary file. 

According to another aspect of the present invention, 
a translation system is also provided. The system 
comprises an electronic collection of user utterances, an 
utterance search unit which identifies a content of a given 
user utterance using the user utterance collection, a 
dictionary file which describes correspondence between 
multiple languages for anticipated utterances of the user, 
a function block which offers a predefined service to the 
user, a target language setting unit which sets a language 
that is used by any number of users who assemble virtually 
to receive the offered service as a target language for 
translation, and a corresponding expression search unit 
which compares a content of an utterance given by any one 
of said users, which is identified by the utterance search 
unit, with the dictionary file and identifies an expression 
corresponding to the utterance in the target language. The 
function block offers the corresponding expression embedded 
in said service . 

The predefined service includes any services in which 
a translation between multiple languages can be utilized. 
The users who assemble virtually to receive the offered 
service may be a plurality of users who join a chat service 
in the Internet by accessing a predefined Web page to 
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receive the service. The language used by such users may 
be their mother tongue. 

In this aspect of the present invention, when a user 
makes an utterance, it is translated to a corresponding 
expression in a target language and embedded into the 
service. Therefore each of the users of the system can 
receive the service in his/her mother tongue. The system 
can be applied to a chat service shared by people speaking 
different languages or a network RPG or role-playing game 
in which many users from different countries can 
participate. The system can be also applied a system that 
only one user uses, such as an on line shopping or a ticket 
booking service offered in a foreign language. 

The function block may customize the service for each 
user on a target language basis, by embedding a 
corresponding expression in each user's mother tongue into 
the service offered to each user. If a user is 
multinational and speaks five languages, there could be 
five corresponding expressions but the service may be 
sufficiently offered to the user in his/her mother tongue 
only. 

According to yet another aspect of the present 
invention, a user support apparatus is provided. The 
apparatus comprises a first block which has an electronic 
collection of user utterances, and identifies a content of 
a given user utterance, and a second block which has an 

SA-70017 



10 Docket 263/123 

electronic collection of action patterns for an agent for 
responding to user utterances, and enables the agent to 
respond to the user utterances. The user utterance 
collection includes a general utterance library that stores 
5 general user utterances and a specialized utterance library 
that stores utterances related to a specialized field of 
the agent . 

The general utterance library may be configured as a 
natural language library such as a dictionary for a kana- 
10 to-kanji converting system in a Japanese word processor. 
It is not necessary to provide the general utterance 
library and the specialized utterance library separately, 
but both the libraries may be united as a one library. 

Another aspect of the present prevention is a user 
15 support system. In the system, a plurality of the user 
support apparatus is provided according to each said 
specialized field, and the plurality of the user support 
apparatus are connected to a network as separate network 
nodes, and each node is so configured as to be accessible 
20 from the user. 

A server that includes the specialized utterance 
libraries of all the user support apparatus within the user 
support system may be provided. This server may be an 
entrance server or a portal server that can identify all 
25 user utterances to be processed at the user support system. 
An appropriate server for responding to the user utterance 
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may be selected according to the content of the utterance 
identified by the server. 

Still another object of the present invention is to 
provide a user support technology by means of which a user 
can get desired information in a friendly environment and 
desired processes can be smoothly executed on a computer or 
other apparatus. 

According to one aspect of the present invention, a 
user support apparatus is provided. The apparatus 
comprises an utterance identification block which has an 
electronic collection of user utterances, and identifies a 
content of a given user utterance, and a response block 
which has an electronic collection of action patterns for a 
first agent for responding to user utterances, and enables 
the first agent to respond to the user utterances. The 
utterance identification block has an additional collection 
of anticipated utterances to which the first agent should 
react among utterances that a second agent make to the 
user, and identifies a content of an utterance of the 
second agent if the utterance of the second agent exists in 
the additional utterance collection, and the response block 
has an additional collection of action patterns for the 
first agent for reacting to the utterances of second agent, 
and enables the first agent to occasionally react to the 
utterances of the second agent. 

The agent here is a generic name of a function for 
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supporting a user to search information or navigating the 
user to access desired information, and the function mainly 
enables a personified character to appear on a screen and 
converse with the user. The first agent is implemented on 
5 the user support apparatus and acts inside the user support 
apparatus, while the second agent may act outside the user 
support apparatus. While the second agent and the user 
converse, the first agent can react if the conversation is 
related to his interests, even when the first agent is not 

10 talking with the user. 

Another aspect of the present invention is a user 
support system. In the system, a plurality of the user 
support apparatus is provided according to each specialized 
field, and the plurality of the user support apparatus are 

15 connected to a network as separate network nodes, and the 
additional utterance collection, the agent action 
collection, and the additional action collection of each 
user support apparatus are generated according to each 
specialized field . 

20 In this system, the plural user support apparatus may 

include the respective response blocks therein and share 
the utterance identification block at any one of the 
network nodes. In this configuration, the shared utterance 
identification block may include the user utterance 

25 collections of all other apparatuses. 

In this system, each user support apparatus may 
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include the first agent on the apparatus, and if the first 
agent appears on any other apparatus, the first agent may 
act as a second agent on said other apparatus. 

Moreover, any arbitrary combination of the above- 
mentioned structural components in the present invention is 
still effective as an embodiment when applied as a method, 
a system, a server, a terminal or a computer program, and 
so forth. 

This summary of the invention does not necessarily 
describe all necessary features, so that the invention may 
also be a sub -combination of these described features. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is an overall structure of a network system 
including a user support system according to the first 
embodiment . 

Fig. 2 is an internal structure of an originating 
server in a user support system. 

Fig. 3 is an internal structure of an index file in 
an originating server. 

Fig. 4 is an internal structure of a user utterance 
collection in an originating server. 

Fig. 5 is an internal structure of an access 
information file in an originating server. 

Fig. 6 is an internal structure of a chat server in a 
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user support system. 

Fig, 7 is an internal structure of a user terminal to 
utilize a user support system. 

Fig, 8 shows a local agent displayed on a screen when 
a user has activated a user terminal. 

Fig. 9 shows a chat agent displayed on a screen when 
a user makes an utterance . 

Fig. 10 shows a menu agent displayed on a screen when 
a user asks for a recipe. 

Fig. 11 shows how a recipe agent requests a user to 
give a hint to narrow a search scope . 

Fig. 12 shows how a recipe agent presents a search 
result to a user. 

Fig. 13 is an overall structure of a network system 
including a translation system according to the second and 
the third embodiments . 

Fig. 14 is an internal structure of an entrance 
server as the first block of the second embodiment . 

Fig. 15 is an internal structure of a user utterance 
collection in an entrance server. 

Fig. 16 is an internal structure of a translation 
server as the second block of the second and the third 
embodiments . 

Fig. 17 is a data structure of a dictionary file in a 
translation server . 

Fig. 18 shows a screen displayed when a user accesses 
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an entrance server to receive a translation service. 

Fig. 19 shows a screen displayed when a user accesses 
a translation server. 

Fig. 2 0 is a flow chart showing a translation 
procedure according to the second embodiment . 

Fig. 21 is an internal structure of an entrance 
server according to the third embodiment . 

Fig. 22 illustrates a Japanese version of a screen of 
a multilingual chat service according to the third 
embodiment . 

Fig, 23 illustrates an English version of a screen of 
a multilingual chat service according to the third 
embodiment . 

Fig. 24 is an overall structure of a network system 
including a user supporting system according to the fourth 
embodiment . 

Fig. 25 is an internal structure of an originating 
server in a user support system. 

Fig. 26 is an internal structure of an index file in 
an originating server. 

Fig, 27 is an internal structure of a user utterance 
collection in an originating server. 

Fig. 2 8 is an internal structure of an access 
information file in an originating server. 

Fig, 2 9 is an internal structure of an agent action 
collection in an originating server. 
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Fig, 3 0 is an overall structure of a network system 
including a user support system according to the fifth 
embodiment . 

Fig, 31 is an internal structure of an originating 
5 server in a user support system. 

Fig, 3 2 is an internal structure of an additional 
index file in an originating server. 

Fig. 33 is an internal structure of an additional 
utterance collection in an originating server. 
Jy{ 10 Fig. 34 shows a local agent displayed on a screen 

m when a user has activated a user terminal. 

ly Fig. 3 5 shows a chat agent displayed on a screen when 

i 

i|| a user makes an utterance. 

y Fig. 3 6 shows a recipe agent displayed on a screen 

2 15 when a user asks for a recipe. 

J Fig. 37 shows how a recipe agent presents a search 

result to a user. 

Fig. 38 shows a travel agent displayed on a screen. 
Fig, 3 9 is an overall structure of a user support 
20 apparatus according to the sixth embodiment. 

Fig, 40 is an overall structure of a user support 
apparatus according to the seventh embodiment . 

DETAILED DESCRIPTION OF THE INVENTION 

25 



The invention 



will now be described on the basis of 
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the preferred embodiments, which do not intend to limit the 
scope of the present invention, but exemplify the invention. 
All of the features and the combinations thereof described 
in the embodiment are not necessarily essential to the 
5 invention. 

Fig. 1 shows an overall structure of a network system 
10 including a user support system 16 according to the 
first embodiment of the present invention. Here a user 
terminal 12 and a user support system 16 are connected to 
;S 10 each other via the Internet 14 . The user terminal 12 is a 

III personal computer, a PDA or personal digital assistant, a 

ilj mobile phone with access to the Internet 14, or any other 

llJ suitable item of hardware . 

Q The user support system 16 includes an originating 

^ 15 server 20, a chat server 24 and a recipe server 26. The 

•r^ originating server 20 is the first processing block, and 

both the latter two servers are the second processing 
blocks . These three servers are connected to the Internet 
14. Thus, the originating server 20, the chat server 24, 
20 and the recipe server 2 6 are configured as separate network 
nodes, and therefore the processes of user utterances and 
agent utterances can be distributed among the servers. 
Since an agent performing in a different field can be also 
implemented in a different node, maintenance can easily be 
25 carried out on each of the agents. The names "chat server" 
and "recipe server" are given according to an allotted 
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field or a specialized field of the agent. In the 
following, such servers as the chat server 24 and the 
recipe server 26 are each generally referred to as a 
specialized server, and agents placed on these servers are 
referred to as expert agents. Although the user support 
system 16 may be configured as one unit or apparatus, for 
instance as one component inside a portal site, it is 
assumed in the following that the system is configured as 
separate nodes and the originating server 2 0 serves as a 
portal server for the user terminal 12 . 

Although full details are given below, the abstract 
of the process in Fig. 1 is as follows. When the user 
activates the user terminal 12, a local agent implemented 
inside the user terminal 12 appears on its screen. The 
local agent waits for the first utterance of the user. 
This utterance is referred to as a process starting 
utterance in the following. The process starting utterance 
is transmitted to the originating server 20 via the 
Internet 14. At that time, the user terminal 12 displays a 
Web page of the originating server 2 0 on a WWW browser. 

The originating server 2 0 has a collection of user 
utterances, that is a collection of utterances that users 
are expected or anticipated to produce. The process 
starting utterance is matched with the collection and the 
content of the utterance is recognized. As a result, an 
expert agent appropriate to respond to the process starting 
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Utterance is identified and the URL of its specialized 
server, as denoted by URLa and URLb in the figure, is sent 
to the browser of the user terminal 12 . When the user 
terminal 12 obtains the URL, a Web page of the specialized 
server is displayed on the screen, and the expert agent 
appears. The specialized server contains a collection of 
action patterns for the expert agent, and responds to the 
process starting utterance and subsequent user utterances, 
which are referred to as normal utterances . Although 
utterances of the agent are mainly considered as the agent 
behavior in the following, the agent may respond to the 
user through a gesture or other actions, or may respond by 
changing the color or texture of its image, or performing a 
search or any other program processes. 

When the user makes a new utterance, that is a normal 
utterance, to the expert agent, the utterance is captured 
and sent to the originating server 20, as denoted by URLs 
in the figure. The originating server 20 identifies again 
an expert agent to respond to the utterance, and then 
transmits the URL of its specialized server to the user 
terminal 12. Again, the following sequence is repeated: 

1. the originating server 20 identifies a user 
utterance ; 

2. the originating server 20 identifies a specialized 
server appropriate to the identified utterance; 
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3. an expert agent on the specialized server 
responds to the user; and 

4. the expert agent requests or prompts the user to 
make a normal utterance . 

Thus, the process always returns to the originating 
server 20 and then restarts from there. It is for this 
reason that the server is named the originating server. 

Fig. 2 shows an internal structure of the originating 
server 20. In this figure, "H" indicates utterance data, 
"I" an index search of the utterance, "F" a file name 
having the URL of a specialized server to respond to the 
utterance of the user, and "X" an unidentified utterance, 
respectively. The structure shown in Fig, 2 may be 
implemented with a CPU, memory and a program loaded in the 
memory. In the figure, however, the blocks are not divided 
in terms of hardware and/or software components, but in 
terms of function. The skilled in the art can therefore 
understand that the various combinations of hardware and 
software components can achieve the function of these 
blocks. The same consideration is applied to the whole 
specification . 

A communication unit 3 0 communicates with the 
specialized server and the user terminal 12 via the 
Internet 14. An utterance obtaining unit 32 captures an 
utterance from a user and sends it to an utterance search 
unit 34. The utterance search unit 34 initially checks the 
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first character of the utterance with an index file 36 to 
search by index, and then identifies the content of the 
utterance by conducting a phrase search through the whole 
utterance. The phrase search is a process of finding any 
phrase that matches the utterance not only by word but also 
by phrase. If no corresponding phrase is found, the 
utterance is divided into morphemes and a closely related 
expression is searched for by key word or word. 

The index file 36 is generated by arranging the 
anticipated utterances stored in a user utterance 
collection 3 8 in the order of the Japanese syllabary. 
Since the first character of the utterance is checked with 
this index file 36, the search for the utterance can be 
conducted with great speed, even if the user utterance 
collection 3 8 is very large. As described below, since the 
user utterance collection can easily be enhanced in this 
embodiment, the utterance collection 3 8 can be greatly 
increased in size. In this respect, the speed gained by 
the initial index search is highly advantageous. 

When an utterance is identified using the index file 
36, a file descriptor of a file describing information such 
as a URL of a specialized server that should respond to the 
utterance is identified in the index file 36, and the file 
itself built into the user utterance collection 38 is 
opened and the proper URL obtained. The user utterance 
collection 3 8 has one file devoted to each utterance. The 
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URL obtained is forwarded to the browser of the user 
terminal 12 via the communication unit 3 0 and the user 
terminal 12 in turn accesses the specialized server. 
Strictly speaking, the URL does not point to a general Web 
5 page of the specialized server, but a personalized page to 
respond to the utterance of the user. One page is 
allocated to one utterance, and in some cases, multiple 
pages are allocated to one utterance. The latter cases are 
described below, 

10 A statement exactly corresponding to the utterance of 

the user may not always have been previously stored in the 
user utterance collection 38. Especially in the process of 
enhancing the user utterance collection 38, a perfectly 
corresponding statement may not be found. In this case, 

15 the utterance search unit 34 breaks down the user utterance 
into morphemes by a known method and finds the most 
probable utterance from the user utterance collection 3 8 by 
re-searching employing a logical AND of nouns of morphemes 
or similar processes. Each utterance for which a re-search 

20 is conducted and each utterance for which the re- search is 
not successful is recorded as an unidentified utterance in 
an unidentified utterance file 40, and an administrator of 
the originating server 2 0 is notified of this via the 
communication unit 42 in an electronic mail or the like. 

25 The administrator registers anew such unidentified 

utterances and the URL of a page of a specialized server 
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that should respond to the utterance in the user utterance 
collection 38^ and registers the indexes of the utterance 
in the index file 36, and then finally designs processes 
including utterances for the expert agent on that page . 
5 For this kind of maintenance, the unidentified utterance 
can be added straight to the user utterance collection 38 
and no complicated process is involved. Therefore it is a 
very easy task to enhance the user utterance collection 38. 
An access record unit 44 records the status of each 

10 user's accessing of the specialized server in an access 
information file 46. This enables the expert agent to 
respond differently to identical user utterances. For 
instance, when a user who first visits the chat server 24 
says "Hello", the expert agent of the chat server 24, also 

15 referred to as a chat agent, will say "Nice to meet you". 
On the other hand, if the user visits the chat server 24 
again, the chat agent can say "Hello. How's it going?" and 
so on. Therefore, a certain sensitivity of response can 
be realized. The access record unit 44 notifies the 

20 utterance search unit 34 of the user's access status. If 
multiple pages of the specialized server are employed in 
the user utterance collection 3 8 in order to respond to a 
user utterance, as in this example, the utterance search 
unit 34 chooses an appropriate page under the user access 

25 status and sets the URL of the chosen page on the browser 
of the user terminal 12. 
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Fig. 3 is an internal structure of the index file 36. 
Fig. 4 is an internal structure of the user utterance 
collection 38. The index file 36 has a Japanese syllabary 
column 100, a user utterance column 102, and a file name 
5 column 104. The user utterances are arranged in the order 
of the Japanese syllabary. If the first character is "A", 
the utterance is categorized corresponding to "A" of the 
Japanese syllabary column 100. Likewise, the utterances 
are categorized by using the first character as shown in 

10 the figure. 

The user utterance collection 38 has a file name 
column 104, a user utterance column 102, and a page column 
12 0 of a specialized server to respond to the user. For 
instance, a page of a specialized server to respond to the 

15 utterance "Hi" is URLa43, and a pair of the utterance "Hi" 
and URLa43 forms a file f044. The user utterances are 
gathered for each specialized server. For instance, the 
user utterances 110 which are linked to the chat server 24 
are put together into one group, while the user utterances 

20 120 linked to the recipe server 2 6 are put together into 
another group. The former utterances relate to general 
greetings and such. The latter utterances relate to 
cooking and recipes. The index file 3 6 and the user 
utterance collection 38 are linked together via file names. 

25 For instance, the file name f04 5 is recorded corresponding 
to the utterance "Hello" in the index file 36, and the file 
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name points to the file f045 in the user utterance 
collection 38 • 

As shown in Fig. 4, two pages ^ URLal and URLa2, 
correspond to "Hello" . URLal will be sent to a user who 
5 first visits the chat server 24 and URLa2 is sent to a user 
who visits the server a further time. 

Fig. 5 illustrates an internal description of the 
access information file 46. In this figure, the user 
"userl" has visited the specialized servers called "chat", 

10 "recipe"^ and "auction" before, while the user "user2" has 
already visited the specialized servers named "travel" and 
"PC". Therefore, as stated above, when "user2" visits the 
chat server 24, the chat agent starts with an utterance 
prepared for first-time visitors. When "userl" visits the 

15 chat server 24, the chat agent produces an utterance 
prepared for revisitors. 

Fig. 6 is an internal structure of the chat server 24 
as an example of a specialized server. A communication 
unit 6 0 communicates with the user terminal 12 and the 

20 originating server 20 via the Internet 14, The URL 
identified in the utterance search unit 34 of the 
originating server 20, for instance, URLal or URLa2 
corresponding to the utterance "Hello" as in Fig. 4, is 
forwarded to an agent behavior collection 62 via the 

25 communication unit 60. The agent behavior collection 62 
includes agent data 72 that describe images and action 
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patterns of the expert agent as well as its utterances. 
One page corresponding to one URL identified by the 
utterance search unit 34 is also provided. For instance, a 
page 64 for URLal, a page 66 for URLa2, and a page 6 8 for 
5 URLan are provided. The pages are Web pages that not only 
carry the utterances of the chat agent, but also display 
its image and behavior, and perform services using the 
agent, for instance for information retrieval and such. 
Thus, by providing one Web page for each single utterance, 

10 fully flexible responses can be realized. 

Each page has almost the same configuration, so only 
page 64 of URLal is described in detail in this figure. 
Page 64 of the URLal has an agent output unit 70, a user 
utterance obtaining unit 74, a specific process execution 

15 unit 76. These units are realized by using CGI or Common 
Gateway Interface that is a back-end process behind this 
page. The agent output unit 70 responds to the user 
utterance through the chat agent on the basis of the agent 
data 72. The specific process execution unit 76 performs 

20 any processes other than that of responding to utterances, 
for instance, retrieving information and executing various 
types of programs. For example, if the user utterance that 
brought the user to access this page is "I want to know 
today's news", the chat agent will search news through the 

25 Internet 14 and present it to the user. The user utterance 
obtaining unit 74 thereafter obtains a normal utterance 
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from the user, and notifies the originating server 2 0 of 
this. As a result, a new specialized server is identified 
by the originating server 20. 

Fig. 7 shows the internal structure of the user 
terminal 12, A communication unit 13 0 communicates with 
the originating server 20, the chat server 24, the recipe 
server 26, and other specialized servers via the Internet 
14. A user interface 13 8 is a general term for the whole 
structure used to encourage a user to make a decision and 
enabling the user to input his/her decision, and it 
includes a keyboard, a mouse, a display, and other types of 
data interfaces. A local agent output unit 132 reads local 
agent data 134 and forwards the data to the user via a user 
interface 13 8. The process starting utterance and normal 
utterances of the user are forwarded to a user utterance 
input unit 13 6 and these data are sent to the originating 
server 2 0 via the communication unit 13 0 and the Internet 
14. The processes involved in the above-mentioned 
configuration of the embodiment are now described using 
some examples as follows. 

Fig. 8 shows a screen 150 displayed when a user has 
activated the user terminal 12. A local agent 152 appears 
and says, "Welcome 1 Let's chat." The user inputs "Hello" 
in an input field 154 and presses a send button. The 
screen may be configured in such a manner that the input 
field 154 appears when the user clicks the local agent 152. 
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In this case, as long as the user does not click, the local 
agent 152 may continue chatting or encourage the user to 
talk by asking a question. In any case the inputted 
statement "Hello" is sent as a process starting utterance 
to the originating server 20, and the chat server 24 is 
identified as a specialized server on the basis of the 
content of the statement, and the user terminal 12 is given 
access to a corresponding page. 

Fig. 9 shows a screen 15 0 displayed when the user 
makes an utterance. Here a chat agent 156 appears, but the 
same image as the local agent 152 is used in this 
embodiment and thus the conversation continues with no 
apparent seams. The chat agent 156 says, "Hello. I am a 
chat agent. Call me Peako." When the user inputs 
"Recommend a recipe" and sends it, the utterance is 
obtained at the originating server 2 0 and a page of the 
recipe server 2 6 is anew identified. The URL of the 
identified page is sent to the user terminal 12 and the 
user terminal 12 is given access to the page. 

Fig. 10 shows a screen 150 displayed when the user 
asks for a recipe. A new recipe agent 160 appears and says, 
"OKI I am a recipe agent. Trust me." Then the agent asks, 
"What kind of recipe would you like?" The reason for this 
is that there could be many choices when a user asks for a 
recipe and it is necessary to obtain a hint to narrow the 
scope of the search. The user inputs "Chinese cooking" and 
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submits it. Although this normal utterance is sent to the 
originating server 20, the specialized server identified 
therein is still the same recipe server 26 and the user is 
given access to another page of the same server. 
5 Fig. 11 shows a screen 150 displayed when the user 

gives a hint to narrow down the search. Here the recipe 
agent 160 asks, "Now, how about ingredients? Please select 
one" in order to specify the search further. A region 162 
appears at the lower part of the screen 150, displaying 
10 several choices of ingredients, with radio buttons. Pork, 
beef, fish and such are given as choices. Let us say that 
the user selects "beef" and presses a send button. 

Fig. 12 shows a screen 150 that displays a search 
result. Although the screen 150 of Fig. 12 is different 
15 from the screen 150 of Fig. 11, this is not because the 

accessed page has changed in response to the utterance, but 
because a page linked to the radio button "beef" has been 
accessed. Thus, generally speaking, by embedding 
predefined choices in the utterances of the expert agent, 
20 the user's intentions can be easily and reliably confirmed. 

In Fig. 12, the recipe agent 160 says, "How about 
this?" and some recipe titles 170 such as "Fried beef with 
green peppers" are displayed, since the search on the 
Internet 14 has already been performed by the specific 
25 process unit 76 under the query criteria "Chinese dishes" 
AND "beef" AND "recipe", that correspond to the user's 
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request. These titles have links through which the user 
can access Web pages describing the recipes in detail. In 
addition, other Web sites offering Chinese recipes are 
displayed in a research result field 172 for the 
convenience of the user. In any case, the user can obtain 
the details of the recipes by starting from this screen 
150, In this figure, another normal utterance of the user 
"I would like to look at flight schedules" is inputted in 
an input field 154. When it is sent, the originating 
server 2 0 identifies a travel agent on a travel server that 
is not shown in the figure and the necessary processes are 
initiated. 

On the other hand, if the user inputs "I want to know 
about Egyptian palace dishes", this utterance may not be 
identified. In such a case, a system administrator is 
notified of the unidentified utterance as it is, and then 
the user utterance collection 38 and the index file 36 are 
updated. In addition, a new page to respond to the 
utterance is provided in the recipe server, and thereby 
information regarding Egyptian palace dishes is properly 
returned . 

Although the present invention has been described by 
way of exemplary embodiments, it should be understood that 
those skilled in the art might make numerous changes and 
substitutions without departing from the spirit and the 
scope of the present invention that is defined by the 
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appended claims. Some such alterations are stated as 
follows , 

Although the user utterance is performed on a text 
basis in the embodiment, it may be performed using speech 
recognition. The agent may also make utterances in voice. 

Although the unidentified utterance is considered as 
an utterance that is not identifiable in the user utterance 
collection 38, if the utterance is identifiable in the user 
utterance collection 38 but the response of the expert 
agent is not complete or fails, the utterance may be called 
an unidentified utterance. For instance, when the specific 
process execution unit 76 searches for a user utterance 
"Recommend a recipe" and the search results are too many to 
satisfy the user, the utterance may be reported to the 
system administrator as an unidentified utterance so that 
the response of the expert agent can be improved. 

In the embodiment, the expert agent utterance is 
appropriately selected according to the record of the 
user's access to the specialized server. Moreover, an 
appropriate utterance of the agent may be selected based on 
the user attributes. For instance, if the user is female, 
a relatively gentle expression may be chosen or if the user 
is an elder, a polite expression may be chosen. 

Although the local agent 152 and the chat agent 156 
have the same image in the embodiment, it is not necessary. 
The local agent 152 may be implemented on the originating 
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server 20 instead of the user terminal 12 as a process- 
initiating agent, for instance. 

The second embodiment of the present invention is now 
explained. This embodiment relates to a translation 
technique, and it particularly relates to a translation 
technique using a server-client system or other systems. 

There has been a well-known apparatus called an 
electronic dictionary in an electronic calculator. The 
apparatus displays an English word or other foreign words 
corresponding to a word inputted by a user. The 
conventional dictionaries in print are very heavy, while 
portable dictionaries have very small characters. Both are 
generally inconvenient for traveling on business or on 
holiday. However, the electronic dictionary has 
portability and is suitable for traveling abroad, and it 
can also save a space for a use at home. Therefore it has 
been gaining support among specific users. 

Besides such a dictionary apparatus, various types of 
dictionaries are on sale as computer software. Although 
its portability depends on the hardware of the personal 
computer, such software made the dictionaries easier to 
use. For this reason, not only users who write documents 
in foreign language but professional translators use such a 
software package of dictionaries. 

Such apparatus and software are designed for the use 
of looking up a word in dictionaries. The dictionary used 
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in the apparatus or the software package is originally just 
an electronic version of a dictionary in print, and the 
dictionary is edited on a word basis. 

Apart from professional translators and users who 
write/speak in a foreign language, general users find very 
difficult to form words into a sentence when they look up 
the words in a dictionary. In that sense, the current 
electronic dictionaries and software packages are not a 
true communication tool, although it is only natural 
considering their purpose. 

Fig. 13 shows an overall structure of a network 
system 3 010 including a translation system 3016 according 
to the second embodiment of the present invention. Here a 
user terminal 3 012 and the translation system 3 016 are 
connected to each other via the Internet 3 014. 

The translation system 3 016 includes an entrance 
server 3020 and a translation server 3024. The entrance 
server 3020 and the translation server 3024 may be 
configured as a united node within one site or may be 
configured as separate nodes. When the servers are 
separately configured, the processes can be distributed 
among the servers resulting in a balanced or optimized load, 
and additionally the system maintenance becomes easier. 
When the servers are configured into one body, the total 
hardware resources of the system can be saved. It depends 
on the system design and the system management policy which 
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configuration should be chosen. 

In the figure, URLa and URLb denote respectively a 
network address of the entrance server 3 02 0 and the 
translation server 3024. A user first accesses the 
5 entrance server 3 02 0 in order to receive a translation 

service. The entrance server 3020 receives an utterance of 
the user and identifies the content of the utterance. When 
the content of the utterance is identified, the entrance 
server 3020 sets the address URLb of the translation server 

10 3024 to the user terminal 3012 and the user terminal 3012 
comes to access the translation server 3024. The 
translation server 3 024 translates the content of the 
identified content of the utterance and sends the 
translated content back to the user. Next the translation 

15 server 3 024 receives another utterance from the user and 
sends it to the entrance server 3 020. At this time, the 
translation server 3 024 sets the address URLa of the 
entrance server 3 02 0 to the user terminal 3 012 so that the 
user comes to access the entrance server 3 02 0 again. 

20 Likewise the processes are repeated between the entrance 
server 3020 and the translation server 3024. 

Fig. 14 shows an internal structure of the entrance 
server 3020. The entrance server 3020 may be an Internet 
service site such as a Web server or the like. In this 

25 case, the entrance server 3 02 0 can be configured in various 
manners such that the main functions remains at the server 
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side like CGI or Common Gateway Interface, the main 
functions are transferred to the client side like a Java 
(trademark) applet or ActiveX (trademark) , and an API or 
Application Program Interface type, that is, the main 
functions are provided at both the server and the client 
like a Java application. The translation server 3 024 is 
the same as the entrance server 3020 in this respect. In 
Fig. 14, "H" indicates an utterance, "TL" a target language 
for translation, "R" a record of database, "UW" an 
unidentified utterance, that is an utterance the content of 
which is not identified respectively. 

A communication unit 3 03 0 of the entrance server 3 02 0 
communicates with the user terminal 3 012 and the 
translation server via the Internet 3014. An utterance 
obtaining unit 3 03 2 obtains an utterance inputted by the 
user and sends it to an utterance search unit 3034. The 
utterance search unit 3 034 looks up the utterance in a user 
utterance collection 3 03 8 and identifies the content of the 
utterance. This identification is conducted by finding a 
sentence corresponding to the user utterance as a whole. 
The identified utterance is sent to the translation server 
3 024 in the form of record number R, and a corresponding 
record in a dictionary file built in the translation server 
3 024 is retrieved. After identifying the utterance, the 
utterance search unit 3 034 sets the address URLb of the 
translation server 3024 to the user terminal 3012. 
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When a sentence completely corresponding to the 
utterance of the user is not stored in the user utterance 
collection 3038, an almost similar process is conducted as 
in the first embodiment. Only differences are described 
here. The utterance for which a re-search is conducted or 
the utterance for which the re-searching is not successful 
are recorded as an unidentified utterance in an 
unidentified utterance file 3 040, and an administrator of 
the translation system 3 016 is notified of this via a 
reporting unit 3042 using an electronic mail or the like. 
The administrator newly registers such an unidentified 
utterance and a corresponding expression in the user 
utterance collection 3 03 8 and the dictionary file, and then 
finally improves the system. 

A target language setting unit 3 044 obtains a target 
language, that is a translation language specified by the 
user, and passes it to the translation server 3024. The 
translation server 3024 thereby performs translation in a 
language that the user desired among many languages. 

Fig. 15 is an internal structure of the user 
utterance collection 3038. The user utterance collection 
3038 has a first character column 3050, a user utterance 
column 3052, and a record column 3054. The Japanese 
syllabary is written in the first character column 3 050 and 
assumed utterances of users are arranged in the order of 
the Japanese syllabary in the user utterance column 3052. 
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The user utterance may be a word such as "Asa", or may be a 
conversational sentence or the like. The record column 
3 054 has a record number R of the dictionary file of the 
translation server 3 024, and therefore it becomes an index 
5 for referring to the dictionary. In the figure, the index 
of the user utterance "Arigato" that means "Thank you" is 
R112, for instance. 

Fig. 16 is an internal structure of the translation 
server 3024. A communication unit 3060 communicates with 
10 the user terminal 3012 and the originating server 3020 via 
the Internet 3014. The record number R identified at the 
utterance search unit 3 034 of the entrance server 3 02 0 is 
received at a record obtaining unit 3062 via a 
communication unit 306 0 and passed to a corresponding 
15 expression search unit 3064. On the other hand, the target 
language obtained at the target language setting unit 3 044 
of the entrance server 3020 is also passed to the 
corresponding expression search unit 3064. The 
corresponding expression search unit 3064 retrieves a 
20 corresponding expression, as indicated by SR in the figure, 
in a dictionary file 3 066 based on the given information, 
and passed it to a formatting unit 3068. The formatting 
unit 3 068 changes the corresponding expression into a Web 
page or an electronic mail form, as indicated by P in the 
25 figure, and sends it to the user terminal 3 012 via the 
communication unit 3060. An instruction obtaining unit 
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3 070 obtains the next user utterance and sends it to the 
entrance server 3020, or receives a user's choice when a 
plurality of corresponding expressions exists, or receives 
a user's instruction to paste the corresponding expression 
in an document or an electronic mail under editing, or 
performs any other processes. The instruction obtaining 
unit 3070 ends up with causing the user terminal 3012 to 
access the entrance server 3020, namely URLa, in order to 
identify the next utterance. 

Fig. 17 is a data structure of the dictionary file 
3066. The dictionary file 3066 has a record column 3054, 
an English column 3080, a French column 3082, and other 
language columns that are not shown in the figure. For the 
user utterance "Arigato" with a record number R112, two 
choices are shown here respectively as its English 
translations and French translations. For instance, "Thank 
you" for a normal expression and "Thank you very much" for 
a polite expression are on the list of English translations 
so that users can select one depending on situations. 

Now explain a flow of a translation service by the 
above-mentioned configuration. Fig, 18 shows a screen 
displayed when a user accesses the entrance server 3020 to 
receive a translation service. There is a field 3090 for 
the user to specify a translation language or a target 
language under the service title "VIRTUAL TRANSLATOR" and 
English is selected here. Under this field an utterance 
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input field 3 0 92 is provided with a statement "Please input 
a sentence or word to be translated" . The user inputs 
"Arigato" and presses a send button 3094. A series of 
processes by the entrance server 3 020 are executed by this 
5 user action, and the target language, that is English, and 
a record number R112 are sent to the translation server 
3024. At the same time, the access destination of the user 
terminal 3012 changes to the translation server 3024. 
Fig. 19 shows a screen displayed when the user 
Q 10 accesses the translation server 3024. The translation 

te server 3 024 identifies the records of expressions 

corresponding to "Arigato" based on the record number R112 

If ^? 

JfJ and provides a desired translation based on the target 

PI language "English" sent from the entrance server 3020. Two 

|d 15 translations 3 096 corresponding to "Arigato" are shown with 

O simple explanations as described above. If the user 

selects one from these translations by clicking, various 
processes can be performed by the instruction obtaining 
unit 3 070 of the translation server 3 024, such as pasting 
20 the translation into an electronic mail under editing. 

Fig. 20 is a flow chart showing the above-mentioned 
processes. A user sets a target language (S3010) . The 
utterance obtaining unit 3 032 obtains a user utterance 
(S3012) . The utterance search unit 3034 searches the 
25 utterance (S3014) . If the user utterance is found in the 
user utterance collection 3038 (Y of S3016) , the 
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translation server 3024 translates it (S3018) , formats it 
(S3020) , and then provides it to the user terminal 3012 
(S3022) . On the other hand, if the user utterance is not 
found in the user utterance collection 3038, and its 
5 content cannot be identified even by a morpheme analysis 

and the like (N of S3 016) , the utterance is recorded in the 
unidentified utterance file 3040 (S3024) and the system 
administrator is notified of it (S3026) . As mentioned 
above, the translation system of the second embodiment can 
10 realize a translation to meet user's requirements, while 

maintaining a high responsiveness and improving the ease to 
maintain . 

The third embodiment of the present invention is now 
explained. The translation system of the present invention 

15 can be applied to other types of applications that a 
plurality of users get involved in. Fig. 21 shows an 
entrance server 3020 for realizing a chat room where users 
from many countries assemble. In the figure, the same 
numerals are given for the components equivalent to ones of 

20 Fig. 14 and the explanations are omitted if appropriate. 

A chat function block 3102 provides functions to 
realize a general conventional chat room, such as 
registering a user's handle or nickname, entering a room, 
administrating users who entered a room, obtaining users' 

25 chat, and broadcasting the chat. The chat function block 
3102 has a user attribute obtaining unit that is not shown 
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in the block, and the unit obtains information indicating a 
user's mother tongue, as referred to as mother tongue 
information, as well as normal user information. The 
mother tongue information may be inputted by a user or it 
5 may be detected which language is used to display Web pages 
on the user terminal 3012. Or the mother tongue may be 
judged by terminal information such as OS installed on the 
user terminal 3012. Attribute information of all users who 
entered in the chat room, as referred to as members, as 

10 indicated by "UI" in the figure, especially the user^s 

mother tongue information is transmitted to a mother tongue 
identifier 3106. The mother tongue identifier 3106 
identifies all target languages indicated by "TL" based on 
the mother tongues of their members and send them to the 

15 translation server 3024. 

The chat function block 3102 basically obtains the 
utterance "H" of each member and sends it to the utterance 
search unit 3034. Thus the function of the utterance 
obtaining unit 3 032 of the second embodiment is realized 

20 inside the chat function block 3102. When a record number 
"R" is identified by the utterance search unit 3 034, it is 
sent to the translation server 3024. The formatting unit 
3104 receives a translation result from the translation 
server 3024, and selects a corresponding expression in the 

25 mother tongue of each member by referring to the member's 

attribute information "UI" and formats a displayed page and 
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returns it to the chat function block 3102. The chat 
function block 3102 generates different pages on a member's 
language basis and broadcasts the pages to each member, 
namely, transmits them at the same time. 

The translation server 3024 of this embodiment is 
basically equivalent to one of the second embodiment, but 
the formatting unit 3104 is provided in the entrance server 
3020 in this embodiment as shown in Fig. 21, Therefore the 
translation server 3 024 does not have the formatting unit 
3068 and sends a translation result obtained from the 
dictionary file 3066 to the formatting unit 3104 of the 
entrance server 3 02 0 and the result is formatted there. 

Fig. 22 illustrates a chat room that members with 
different nationality participate in. The names of the 
members such as "torn" are displayed in a member field 3122 
and their conversation progresses in a main field 3120. A 
field 3124 for the member to enter an utterance and a 
submit button 312 6 to send the utterance are provided at 
the bottom. In addition, a "other ]^nguages" button 3128 
is provided. 

Several members such as ken_123 are now chatting with 
each other in the main field 3120. The chat is displayed 
in Japanese and it looks as if all members are Japanese. 
However, "taro" is Japanese, "torn" is American, "Pierre" is 
French, and other members are either Japanese, American or 
French. The screen of Fig. 22 is displayed on the user 
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terminal 3012 of "taro" . 

In this situation, the mother tongue identifier 3106 
identifies "Japanese, English, and French" as the target 
languages. If an utterance of a member is Japanese, the 
content of the utterance is identified by the utterance 
search unit 3034 of the entrance server 3020 and a record 
number R is sent to the translation server 3024. The 
corresponding expression search unit 3 0 64 of the 
translation server 3 024 identifies an English expression 
and a French expression of the Japanese utterance based on 
the record number R and the target language "Japanese, 
English, and French" and the expressions are sent to the 
formatting unit 3104 of the entrance server 3020. The 
formatting unit 3104 transmits page data of the screen of 
Fig. 22 to Japanese members such as "taro" using the mother 
tongue information of each member. 

On the other hand, the formatting unit 3104 transmits 
page data of the screen of Fig. 23 to American members such 
as "tom" . The screen of Fig. 23 is an English version of 
the screen of Fig. 22 and all members looks English- 
speaking people from the member "torn". Likewise, all 
members looks French-speaking people from the member 
"pierre", although it is not shown in the figure. 

The "Other languages" button 3128 is provided on the 
screen as in Fig. 22 and Fig. 23. The button is used when 
the members want to change the screen in other languages 

SA-70017 



44 Docket 263/123 

than their mother tongue. For instance^ when the member 
"taro" presses the button in Fig. 22, an English service 
like Fig. 23 is offered. 

Some alterations are now stated as follows. In the 
5 second and third embodiments, the user utterance is 

performed on a text basis, but it may be performed using 
speech recognition. In this case, the present invention 
may be utilized as an interpretation tool. 

Although the unidentified utterance is considered as 
O 10 an utterance that is not identifiable in the user utterance 

collection 3 038, if the utterance is identifiable in the 
user utterance collection 3 03 8 but the translation is not 
m complete or not satisfactory, the utterance may be called 

p an unidentified utterance. This case is also reported to 

y 15 the system so that the dictionary file 3 066 can be revised, 

O Several choices are found after translation as in Fig. 

19, the formatting unit 3068 and other units may select an 
appropriate expression based on the user attributes. For 
instance, if the user is female, a relatively gentle 
20 expression may be chosen or if the user is an elder, a 
polite expression may be chosen. 

Moreover, the system may be so configured that users 
can specify a tone in translation. For example, users may 
specify "very politely" or "friendly" and so on, and the 
25 formatting unit 3068 and other units may select a suitable 
expression, 
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Although the user utterance collection 3 03 8 is 
configured on the assumption that the first user utterance 
is made in Japanese in the embodiments, the user utterance 
collection may be generated for any other languages. After 
5 judging the user's mother tongue, it can be determined 

which language version of the user utterance collection is 
used. 

The translation system of the embodiments can be 
integrated into a user support apparatus or system* Such 
O 10 an apparatus or system may contain an agent to respond to a 

J user's question, and if the agent cannot understand the 

5J user utterance, the translation process of the embodiments 

¥1^ can be used. On the other hand, a collection of the 

13 agent's response patterns may be created for various 

W 15 languages so that the agent can respond to users who speak 

O different mother tongues. Even if the agent utterances are 

produced in multiple languages, other data such as action 
patterns and images can be shared. Therefore, it is not a 
hard task to extend the system for multiple languages. 
20 Although the translation is made between different 

languages in the embodiments, the present invention can be 
applied to a translation between dialects, or translating 
from an archaic to a modern expression, or helping a talk 
between different generations. 
25 Moreover, the present invention can be applied to a 

real time translation service for a telephone conversation 
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in different languages or an international conference. 

The fourth embodiment of the present invention is now 
explained. Fig, 24 shows an overall structure of a network 
system 4 010 including a user supporting system 4 016 
according to the fourth embodiment of the present invention. 

The user support system 4016 includes a chat server 
4 020, a recipe server 4026, and a travel server 4028 each 
of which is connected to the Internet 4014. Thus, the chat 
server 402 0, the recipe server 4 026, and the travel server 
4 02 8 are configured as separate network nodes, and each of 
these servers processes user utterances and agent 
utterances in respect to each specialized field. For 
instance, the chat server 4 020 processes general greetings 
such as "Hello" or the like, and the recipe server 4026 
processes utterances about cooking such as "Recommend a 
recipe", and the travel server 4028 processes utterances 
about travel such as "Recommend a domestic travel". 

In this embodiment, the chat server 4020 is the 
server that is first accessed by the user who utilizes the 
user support system 4016. The chat server 4020 processes 
the initial utterance of the user. In this sense, the chat 
server 4020 is also referred to as "an originating server". 
In the following, such servers as the chat server 4 020, the 
recipe server 4026, and the travel server 4028 are each 
generally referred to as a specialized server, and agents 
placed on these servers are referred to as expert agents . 
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Each of the specialized servers includes the first 
block which has an electronic collection of user utterances, 
and identifies a content of a given user utterance, and the 
second block which has an electronic collection of action 
patterns for an agent for responding to user utterances, 
and enables the agent to respond to the user utterance. 
The user utterance collection includes a general utterance 
library that stores general user utterances, and a 
specialized utterance library that stores utterances 
related to the specialized field of the expert agent. The 
specialized server identifies the general user utterances 
and the utterances related to its specialized field, and 
enables the expert agent to respond to users. 

The user utterance collection of the originating 
server 4020 stores the specialized utterance libraries of 
all specialized servers in the user support system 4016. 
The originating server 4 02 0 receives the utterance that any 
other specialized server cannot identify, and find a 
specialized server that can process the utterance by 
matching it in the user utterance collection. When a 
specialized server that can process the utterance is found, 
the process is taken over by the specialized server, and 
the specialized server continues to process the user 
utterances as long as the user talks about the specialized 
field. 

The abstract of the process in Fig, 24 is almost the 
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same as in the first embodiment, and only differences are 
explained here. 

When the user makes a new utterance, that is a normal 
utterance, to the expert agent, the utterance is captured 
and checked in the user utterance collection of the 
specialized server. When the content of the utterance is 
identified, the specialized server sends a URL to the Web 
browser on the user terminal 4012 to display another page 
within the identical server, and at the same time retrieves 
an agent action from the agent action collection to perform 
a response process . As long as the user normal utterance 
is identifiable at the specialized server, the specialized 
server repeats a sequence of processes of obtaining a user 
utterance, the expert agent's responding to the user by, 
and requesting or prompting the user to make a normal 
utterance . 

When the content of the utterance is not identified, 
the utterance is sent to the originating server 4020, as 
denoted by URLs. The originating server 4020 identifies a 
specialized agent to respond to the utterance by matching 
it with the user utterance collection and sends the URL of 
the specialized agent to the user terminal 4012. Thus, the 
subsequent process is transferred to the specialized server 
and the second block of the specialized server performs 
subsequent response processes . The utterance that cannot 
be identified by the originating server 402 0 is reported to 
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a system administrator as described below. 

Thus each of specialized server converses with the 
user on a topic in each specialized field. Although the 
originating server itself as a chat server converses with 
the user on a topic related to a chat, the server also has 
a function of identifying user utterances on topics related 
to specialized fields of other specialized servers and 
allocating the access destination of the user terminal 4012 
to a specific specialized server. 

The advantage of this configuration is that a third 
party can develop a specialized server for their 
specialized field independently. An administrator of such 
a specialized server gets a general utterance library from 
the system administrator and develops their original 
specialized utterance library and agent action library. 
Therefore, the development of specialized servers for 
various specialized fields is promoted and an overall 
system that covers many specialized fields can be developed 
in a relatively short time. The general utterance library 
may be provided to each specialized server as package 
software, or a library site to offer the general utterance 
library may be provided and the access right to the library 
site may be given to the specialized server. The latter 
case can save an effort to provide the latest package 
software to the specialized servers whenever the general 
utterance library is updated. 

SA-70017 



50 Docket 263/123 

Fig. 25 shows an internal structure of the 
originating server 4020. The originating server 4020 of 
this embodiment has functions similar to that of the first 
embodiment. Only different functions are explained here. 
5 When an utterance is identified using the index file 

4036, a file descriptor of a file describing information 
such as a URL of a specialized server that should respond 
to the utterance is identified in the index file 4036, and 
the file itself built into the user utterance collection 
y 10 4 03 8 is opened and a proper URL obtained. 

J; If the URL obtained points to the specialized server, 

m the URL is forwarded to an agent controller 4060 as an 

W 

III example of the second block. The agent controller 4 06 0 

P forwards the URL to the browser on the user terminal 4 012 

15 via the communication unit 4030 and retrieves an agent 
action corresponding to the URL from the agent action 
collection 4062 and performs the agent action. 

If the URL obtained points to another specialized 
server, the URL is forwarded to the browser on the user 
20 terminal 4012 via the communication unit 403 0 and the user 
terminal 4012 in turn accesses the specialized server. 

When a statement exactly corresponding to the 
utterance of the user has not been previously stored in the 
user utterance collection 4038, the procedure similar to 
25 that of the first embodiment is taken. Here only different 
points are described. 
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When the system administrator is notified of an 
unidentified utterance, he/she reports the content of the 
utterance to an administrator of a specialized server to 
respond the utterance and requests to develop a response 
process of its expert agent. The administrator of the 
specialized server registers anew such unidentified 
utterances and the URL of a page of the specialized server 
to respond to the utterance in the user utterance 
collection 4038 of their own server, and registers the 
indexes of the utterance in the index file 4036, and then 
finally designs processes including utterances for the 
expert agent on that page . After the development is 
completed, the administrator of the specialized server 
notifies the originating server 4020 of a new developed 
utterance, its index, and a URL of the page of the 
specialized server to respond. The administrator of the 
originating server 4020 registers the notified contents in 
the index file 4036 and the user utterance collection 4038. 

Fig. 26 is an internal structure of the index file 
4036. Fig. 27 is an internal structure of the user 
utterance collection 4038. The structure of the index file 
4 03 6 is the same as one of the first embodiment, but two 
separate index files may be provided respectively for the 
general utterance library and the specialized utterance 
library or one index file containing both libraries may be 
provided. The structure of the user utterance collection 
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4 03 8 is almost same as that of the first embodiment and 
here the user utterances 4114 linked to the travel server 
4028 are put together into one group. 

Fig* 28 illustrates an internal description of the 
5 access information file 4046. As in the first embodiment, 
when "user2" visits the chat server 4020, the chat agent 
starts with an utterance prepared for first-time visitors. 
When "userl" visits the chat server 4 020, the chat agent 
produces an utterance prepared for revisitors. 
10 Fig. 2 9 is an internal structure of the agent action 

collection 4062. The URL identified in the utterance 
search unit 4034 of the originating server 4020, for 



ml instance, URLal or URLa2 corresponding to the utterance 

e: 

ig "Hello" as in Fig, 27, is forwarded to an 'agent behavior 

y 15 collection 4062 via the agent controller 4060. The 

O structure of the agent action collection 4 062 is the same 

as that of the first embodiment except it does not contain 
the user utterance obtaining unit 4074. 

A library provider 4 04 8 manages the general utterance 
20 library and provides the general utterance library to the 
administrators of other specialized servers off line or on 
line. For instance, the general utterance library can be 
provided off line as a software package, and can be 
provided on line by offering an access right for a server 
25 that stores the library. In this embodiment, when the 
library provider 4048 receives a request from any other 
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specialized server, it retrieves the general library from 
the user utterance collection 4038 and transmits it to the 
specialized server via the communication unit 4 030. The 
library provider 404 8 may send and receive a specialized 
utterance library as well as the general utterance library. 
For instance, when any other specialized server develops a 
new specialized utterance library, the library provider 
4048 may receive the specialized utterance library and 
register it in the index file 4036 and the user utterance 
collection 4038. Thereby the maintenance of the user 
support system as a whole becomes easier. 

Now explain the structure and behavior of the 
specialized servers other than the originating server 4020, 
such as the recipe server 4026 and the travel server 4028. 
The recipe server 4 02 6 is explained here as an example, but 
other specialized servers are the same. The internal 
structure of the recipe server 4026 is almost the same as 
one of the originating server 4020 of Fig, 25. Therefore, 
a sequence of the process is explained mainly focusing on 
different functions from the originating server 4020. 

First, when the user utterance obtained at the 
originating server 4 02 0 is judged to be processible at the 
recipe server 4026, a URL of a page within the recipe 
server 4026 is forwarded and set to the browser on the user 
terminal. The browser accesses the recipe server 4026 and 
requests the agent controller 4060 to get a corresponding 
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page via the communication unit 4030. The agent controller 
4 06 0 retrieves the corresponding page from the agent action 
collection 4062 and transmits the page to the user*s 
browser and then executes other necessary processes, as the 
5 originating server 4020 does. 

Next the recipe server 4026 waits until the user 
makes the next utterance. When the user makes an 
utterance, as the originating server 4020 does, the 
utterance obtaining unit 4 032 obtains the utterance and the 

10 utterance search unit 4034 identifies the utterance. The 
recipe server 4026 has a general utterance library and a 
specialized utterance library related to cooking and can 
identify general utterances and cooking-related utterances 
of the user, but cannot identify any other utterances. If 

15 the utterance search unit 4034 can identify the user 

utterance, the agent controller 4060 executes response 
processes. If the user utterance cannot be identified, the 
utterance is recorded as an unidentified utterance in the 
unidentified utterance file 4 040, and the reporting unit 

20 4042 sends the utterance to the originating server 4020. 
Then the utterance is matched with the user utterance 
collection 4038 in the originating server 4020 and a 
specialized se3rver that should process the utterance is 
identified, and the specialized server in turn executes the 

25 subsequent process. When the administrator of the recipe 
server 4026 finds a cooking-related utterance among 
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unidentified utterances recorded in the unidentified 
utterance file 404 0, he/she develops a response process to 
the utterance and registers it in the recipe server 4026. 
Thus, the agent processes can be easily enhanced. In this 
5 embodiment, the unidentified utterances are recorded in the 
unidentified utterance file 4040 of each specialized server 
and that of the originating server 4 020, but may be 
recorded in either specialized servers or the originating 
server 4020. 

13 10 The library provider 4048 accesses the library site 

i 

gl that contains the general utterance library at a predefined 

y interval, and obtains the latest general utterance library 

and then register it to the index file 4 03 6 and the user 
utterance library 4038. The library provider 4048 also 
15 sends a new developed specialized utterance library of the 
recipe server 4026 to the originating server 4020. Thus 
each specialized server is independently maintained 
resulting in enhancing the user support system as a whole. 
The internal structure of the user terminal 4012 is 
20 the same as one of the first embodiment . The screen 

displayed on the user terminal 4012 is also the same as one 
exemplified in the first embodiment. However in Fig. 12, 
when another normal utterance of the user "I would like to 
look at flight schedules" is inputted and sent to the 
25 recipe server 4 026, the recipe server 4026 cannot identify 
this utterance in its user utterance collection 4038 and 
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sends it to the originating server 4020 as an unidentified 
utterance. The originating server 4020 identifies the 
travel server 4028 that can process this utterance by 
matching it with its user utterance collection- The travel 
5 server 4 02 8 initiates the subsequent processes. 

Some alterations are now stated. Although the chat 
server 4 02 0 serves an originating server in this embodiment, 
any other specialized server may serve as the originating 
server, or a plurality of originating servers may be 
10 provided. Instead of a specialized server serving as an 
originating server, an originating server may be provided 
apart from specialized servers. Moreover, instead of 
providing an originating server, the utterance that is not 
identified at a specialized server may be circulated among 
15 other specialized servers and it may be inquired whether 
the utterance is processible at each of the specialized 
servers, and then a specialized server that can process the 
utterance may be identified. 

Although the originating server 4 02 0 identifies the 
20 user utterance and sets a URL of a specialized server's 
page to respond to the utterance in this embodiment, the 
originating server 4020 may only identify a specialized 
server to process the user utterance and the specialized 
server may identify the content of the utterance and set a 
25 URL of a corresponding page to the user terminal. Thereby 
the load on the originating server 4 02 0 can be reduced. 
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The fifth embodiment of the present invention is now 
explained. Since the Internet access at home has been 
common recently, WWW (World Wide Web) users are growing 
rapidly. As it is convenient for the users at home to 
access to a huge amount of information from all over the 
world, the number of users is further increasing. It is 
very likely that necessary information that users want 
exists somewhere in the huge number of web sites. However 
the number of web sites or pages has become too large for 
users to find out such desired information. 

Being aware of the above situation, the system 
administrators of portal sites with search engines have 
been trying hard to make their search methods more 
sophisticated by using a directory tree, for example. 
Therefore, the users can efficiently find necessary 
information out of the flood of information using search 
conditions such as logical OR and logical AND within a 
specific topic or category predefined by the portal sites. 

It is, however, extremely difficult for general users 
to exploit highly complicated search conditions, since most 
of them are beginners in computers. In addition, since 
information available on the Web has a complicated 
hierarchy and in some cases it is difficult for users to 
find desired information out of the hierarchy. The 
oversupply of information may spoil its utilization as the 
number of Web sites is still increasing and more and more 
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beginners are joining the Web services. 

Fig. 3 0 shows an overall structure of a network 
system 5010 including a user support system 5016 according 
to the fifth embodiment of the present invention. 
5 The user support system 5016 includes a originating 

server 5020, a chat server 5024, a recipe server 5026, and 
a travel server 5028. These servers are connected to the 
Internet 5014. The originating server 5020 includes an 
electronic collection of users' anticipated utterances and 

10 an utterance identification block that identifies the 
content of a given user utterance. This utterance 
identification block is shared by other servers in the user 
support system, namely, the chat server 5024, the recipe 
server 5026, and the travel server 5028. The chat server 

15 5024, the recipe server 5026, and the travel server 5028 
each include a collection of action patterns of the first 
agent to respond to the utterance and have a response block 
that enables the first agent to respond to the user 
utterance within each server node. 

20 As in the first embodiment, the originating server 

5020, the chat server 5024, the recipe server 5026, and the 
travel server 5028 are configured as separate network nodes. 
In the following, such servers as the chat server 5024, the 
recipe server 5026, and the travel server 5028 are each 

25 generally referred to as a specialized server, and agents 
placed on these servers are referred to as expert agents. 
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The user utterance is sent to the originating server 
5020 and its content is identified in the user utterance 
collection. Then an agent to respond to the utterance is 
identified according to the content and a response process 
is executed by the response block. An agent on the chat 
server 5024, as also referred to as "a chat agent", 
responds to general greetings such as "Hello", and an agent 
on the recipe server 5026, as also referred to as "a recipe 
agent", responds to utterances related to cooking such as 
"Recommend a recipe", and an agent on the travel server 
5028, as also referred to as "a travel agent", responds to 
utterances related to travels such as "Recommend a domestic 
travel" . Each expert agent finds out what kind of 
information the user wants during a talk with the user, and 
supports the user to search appropriate information among a 
large amount of available information. 

In the user support system of this embodiment, while 
the second agent responds to the user, the first agent 
appears and interrupts the dialogue so that a conversation 
between the two agents is initiated. Through the dialogue 
between the agents, the ongoing process can be made visible 
or some alternatives can be presented to the user. Like a 
rapid-fire comic dialogue, such a dialogue gives fun and 
makes users relaxed. 

For instance, the user says, "What's new?" and the 
chat agent responds to the user saying, "A site of a store 
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offering a good dumpling is open," At this time, the 
recipe agent reacts the key word "dumpling" and says, 
"Don't talk about dumplings without me". Then if the user 
is interested in dumplings, the user says, "Tell me a 
recipe for a good dumpling" and a talk with the recipe 
agent is carried on. Thus, when a specific keyword appears, 
in the dialogue, an expert agent specializing in the field 
automatically appears and talks to the user so that the 
process can smoothly continue. While the system waits for 
an utterance from the user, the agents may continue to talk 
to each other. For instance, when the chat agent says, 
"Don't butt in with your joke when I am talking", the 
recipe agent may say back, "Don't say that. Let me join" 
so that it could make the user relaxed. 

The utterances to which the first agent reacts are not 
necessarily utterances related to its specialized field. 
For instance, while the user and the chat agent talk about 
going to see cherry blossoms, the recipe agent may appear 
unexpectedly and interrupt saying, "Pudding rather than 
flowers. Would you like to know how to make a good 
pudding? " 

The abstract of the process in Fig. 3 0 is almost the 
same as in the first embodiment, and only differences are 

explained here. 

When an expert agent responds to the user utterance, 
its content is sent to the originating server 5020, as 
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denoted by URLs. The originating server 5020 includes an 
additional utterance collection, that is, a collection of 
assumed or anticipated utterances to which other agents 
should respond. The agent utterance is matched with the 
additional utterance collection and then an agent that 
should respond to the utterance is identified. If an agent 
to respond to the utterance is not identified, the system 
will wait for the next user utterance. If an agent to 
respond to the utterance, the URL of the specialized server 
of the agent is forwarded to the browser on the user 
terminal 5012, and a response process is performed by the 
expert agent as it is mentioned above and the system waits 
until the user makes the next utterance. 

When the user makes a new utterance, that is a normal 
utterance, to the expert agent, the utterance is captured 
and sent to the originating server 5020. The originating 
server 5020 identifies again an expert agent to respond to 
the utterance, and then transmits the URL of its 
specialized server to the user terminal 5012. Again, the 
following sequence is repeated: 

1. the originating server 5020 identifies a user 
utterance ; 

2. the originating server 5020 identifies a 
specialized server appropriate to the identified utterance; 

3 . an expert agent on the specialized server responds 
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to the user; 

4 . any other expert agent responds to the expert 
agent utterance (if no other expert agents to respond is 
found, this step will be omitted) ; and 

5. the expert agent requests or prompts the user to 
make a normal utterance. 

Thus, the process always returns to the originating 
server 5020 and then restarts from there. 

Fig. 31 shows an internal structure of the 
originating server 5020. The originating server 5020 of 
this embodiment has functions similar to that of the first 
embodiment. Only different functions are explained here. 

In this embodiment, an utterance obtaining unit 5032 
not only captures an utterance from a user and sends it to 
an utterance search unit 5034, but also captures an 
utterance from an expert agent and sends it to the 
utterance search unit 5034. 

An additional index file 5037 is generated by 
arranging the anticipated utterances stored in an 
additional utterance collection 5039 in the order of the 
Japanese syllabary. As it is the case with the index file 
503 6, since the first character of the agent utterance is 
checked with this index file 5037, the search for the agent 
utterance can be conducted with great speed, even if the 
additional utterance collection 503 9 is very large. 

When an utterance is identified using the index file 
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5037, a file descriptor of a file describing information 
such as a URL of a specialized server that should respond 
to the utterance is identified in the index file 503 7, and 
the file itself built into the additional utterance 
collection 5 03 9 is opened and the proper URL obtained. The 
additional utterance collection 503 9, as the user utterance 
collection 5038, has one file devoted to each utterance. 
The URL obtained is forwarded to the browser of the user 
terminal 5012 via the communication unit 5 03 0 and the 
browser in turn displays a response from the expert agent. 

As it is the case with the user utterance, when a 
statement exactly corresponding to the agent utterance is 
not found in the additional utterance collection 5039, the 
utterance search unit 5034 breaks down the agent utterance 
into morphemes by a known method and finds the most 
probable utterance from the additional utterance collection 
503 9 by re- searching employing a logical AND of nouns of 
morphemes or similar processes. Even if a probable 
utterance cannot be found, it is not necessary to notify 
the system administrator of this. It is because the user 
utterance must be responded by any one of expert agents, 
but the agent utterance does not have to be responded by 
another agent. It is also unnecessary to provide a large 
amount of utterances to which agents should respond. It 
will be good enough to provide a certain amount of such 
utterances to such an extent that another expert agent may 
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appear as frequently as it does not interfere with the 
user. In addition, the user may set a level indicating how 
frequently the other agents may respond to the agent 
utterances. In some cases, the user may prohibit the 
5 agents other than the present conversing agent from 
appearing on the screen. 

A library provider 504 8 provides the user utterance 
collection to a third party off line or on line. For 
instance, the user utterance collection can be provided off 

10 line as a software package, but in this embodiment it is 

provided on line via the communication unit 5030. When the 
library provider 5048 receives a request for the user 
utterance collection via the communication unit 5030, it 
retrieves the user utterance collection 503 8 and transmits 

15 it via the communication unit 5030. At this time the index 
file 5036, the additional index file 5037, and the 
additional utterance collection 5039 may be transmitted. 
By offering the user utterance collection, a new 
development by a third party can be promoted and as a 

20 result the system as a whole can be enhanced. 

When the administrator of each specialized server 
develops a new agent action, a URL of a user utterance and 
its corresponding agent action is sent to the originating 
server 5020. The library provider 5048 receives the 

25 information from the specialized server and registers the 
information in the index file 5036, the additional index 
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file 5037, the user utterance collection 5038, and the 
additional utterance collection 5039. Thereby the accuracy 
of the user utterance collection can be improved and the 
agent functions can be enhanced easily. 

The internal structure of the index file 5036, the 
user utterance collection 503 8, and the access information 
file 5046 are the same as those of the first embodiment. 

Fig. 3 2 is an internal structure of the additional 
index file 5037. Fig. 33 is an internal structure of the 
additional utterance collection 5039. The additional index 
file 503 7 has a Japanese syllabary column 5200, an agent 
utterance column 5202, and a file name column 5204. The 
agent utterances are arranged in the order of the Japanese 
syllabary as the user utterances are in the index file 5036. 

The additional utterance collection 503 9 has a file 
name column 5204, an agent utterance column 5202, and a 
page column 5220 of a specialized server to respond. For 
instance, a page of a specialized server to respond to the 
agent utterance "rainy season" is URLa255, and a pair of 
the utterance "rainy season" and URLa255 forms a file f703. 
The agent utterances are gathered for each specialized 
server. The additional index file 5037 and the additional 
utterance collection 503 9 are linked together via file 
names. For instance, the file name f805 is recorded 
corresponding to the utterance "dumpling" in the additional 
index file 5037, and the file name points to the file f805 
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in the additional utterance collection 5039. 

The internal structure of the chat server 5024 as an 
example of a specialized server is the same as that of 
first embodiment. The internal structure of the user 
5 terminal 5012 is also the same as that of the first 
embodiment . 

Fig. 34 shows a screen 5150 displayed when a user has 
activated the user terminal 5012. A local agent 5152 
appears and says, "Welcome! Let » s chat." The user inputs 

10 "Hello*' in an input field 5154 and presses a send button. 
The inputted statement "Hello" is sent as a process 
starting utterance to the originating server 502 0, and the 
chat server 5024 is identified as a specialized server on 
the basis of the content of the statement, and the user 

15 terminal 5012 is given access to a corresponding page. 

Fig. 3 5 shows a screen 5150 displayed when the user 
makes an utterance. Here a chat agent 5156 appears, but 
the same image as the local agent 5152 is used in this 
embodiment and thus the conversation continues with no 

20 apparent seams. The chat agent 5156 says, "Hello. I am a 
chat agent. Call me Peako." When the user inputs "Tell me 
today's new recipes at cooking sites" and sends it, the 
utterance is obtained at the originating server 502 0 and a 
page of the recipe server 5026 is anew identified. The URL 

25 of the identified page is sent to the user terminal 5012 
and the user terminal 5 012 is given access to the page. 

SA-70017 



67 Docket 2 63/12 3 

Fig. 36 shows a screen 5150 displayed when the user 
asks for a recipe, A new recipe agent 5160 appears and 
says, "OKI I am a recipe agent. Trust me" and accesses a 
cooking site that the user has registered and obtains some 
5 new recipes at the site. In order to prevent the user from 
getting bored during the search, the agent says, "I am 
going to search now. Wait a moment, please" to tell that 
searching is being executed. When the search is completed, 
the browser is given access to a page to display a search 
10 result . 

m Fig. 37 shows a screen 5150 displayed the search 

n I 

W result. The recipe agent 5160 says, "Today's new recipes 

;B are Italian dishes. Please click for cooking details", and 

recipe titles 5170 obtained through the search are 
ft 15 displayed. Each of the titles 5170 has a link to a page 

% describing the recipe in detail. The utterance of the 

recipe agent 5160 is sent to the originating server 5020 

and matched with the additional utterance collection 5039. 

As a result, it is found that a travel agent might react to 
20 the word "Italian". The browser in turn accesses a 

corresponding page of the travel server 5 028 and the travel 

agent appears . 

Fig. 3 8 shows a screen 5150 displayed when the travel 

agent appears. The travel agent 5180 says, "What about 
25 joining an Italian gourmet tour to taste genuine Italian 

cuisine?" The user may ignore the travel agent 5180 and 
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continue to talk with the recipe agent 5160, or may 
initiate a talk with the travel agent 5180. 

Thus each expert agent can serve both as the second 
agent that mainly talks with the user and as the first 
5 agent that interrupts the dialogue between the user and the 
second agent , 

Although the utterance identification block is shared 
at the originating server 502 0 in this embodiment, each 
specialized server may include both the utterance 

10 identification block and the response block. In such a 

configuration, both the user utterance collection and the 
agent action collection can be managed independently for 
each specialized field, and the management and maintenance 
of the agent will become easier. In any configurations, a 

15 central server may be provided to process all the 
utterances , 

Fig. 3 9 shows an overall structure of a user support 
apparatus according to the sixth embodiment . The user 
support apparatus of this embodiment is not a server that 

20 offers services via a network, but a stand-alone apparatus 
that offers services to a user who uses this apparatus. 
The components with the same numerals as in the fifth 
embodiment perform similar functions in this embodiment. 
The agent on this apparatus performs functions 

25 similar to the local agent of the fifth embodiment. The 
agent obtains the user utterance inputted through a user 
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interface 5138 at a user utterance obtaining unit 5032, and 
identifies the content of the utterance at the utterance 
search unit 5034. An agent controller 5070 retrieves a 
page corresponding to the user utterance, which is a page 
stored in the agent action collection 5062 of this 
apparatus, and a response process is executed. 

The user may converse with an external expert agent 
via the communication unit 5030. While the user is talking 
with the external expert agent, the response of the expert 
agent is displayed via the user interface 513 8 and is 
captured by an agent utterance obtaining unit 5031. If the 
utterance of the expert agent is matched with an utterance 
recorded at the additional utterance collection 503 9, 
another agent of this apparatus appears and responds to the 
user . 

Fig. 4 0 shows an overall structure of the user 
support apparatus according to the seventh embodiment , 
This user support apparatus is also a stand-alone apparatus 
as one of the sixth embodiment. The different point is 
that the apparatus of this embodiment has the first agent 
53 0 0 and the second agent 53 02 implemented and the 
utterance identification block and the response block are 
symmetrically provided for the first and the second agents. 
The first and the second agents converse with the user as 
the expert agents of the fifth embodiment. While the first 
agent is mainly talking with the user, the second agent 
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timely reacts to the dialogue. On the other hand, while 
the second agent is mainly talking with the user, the first 
agent timely reacts to the dialogue. 

Although the user utterance collection and the 
additional utterance collection are separately provided in 
this embodiment, the additional utterance collection may be 
incorporated into the user utterance collection. In this 
case, the user utterance and the agent utterance may be 
matched with the user utterance collection without 
discriminating between the two, when the process of the 
utterance identification block comes to at least a final 
stage . 



SA-70017 



